1. Field of the Invention
The invention relates generally to storage clusters and more specifically to systems and methods for simpler, low-cost scale-out storage cluster configurations.
2. Related Patents
This patent application is related to co-pending, commonly owned U.S. patent application Ser. No. 10/329,184 (filed Dec. 23, 2002; the “'184 application” now published as Publication No. 2004/0123017), U.S. patent application Ser. No. 10/328,672 (filed Dec. 23, 2002; now published as Publication No. 2004/0122987), and U.S. patent application Ser. No. 10/671,158 (filed Sep. 25, 2003; now published as Publication No. 2005/0071546), all of which are hereby incorporated by reference. Additionally, U.S. Pat. No. 6,173,374 (issued Jan. 9, 2001) and U.S. Pat. No. 6,073,218 (issued Jun. 6, 2000) provide useful background information and are hereby incorporated by reference.
3. Discussion of Related Art
A mass storage system is used for storing user and system data in data processing applications. A typical mass storage system includes a plurality of computer disk drives configured for cooperatively storing data as a single logically contiguous storage space often referred to as a volume or logical unit. One or more such volumes or logical units may be configured in a storage system. The storage system, therefore, performs much like that of a single computer disk drive when viewed by a host computer system. For example, the host computer system can access data of the storage system much like it would access data of a single internal disk drive, in essence, oblivious to the substantially transparent underlying control of the storage system.
Mass storage systems may employ Redundant Array of Independent Disks (“RAID”) management techniques, such as those described in “A Case For Redundant Arrays Of Inexpensive Disks”, David A. Patterson et al., 1987 (“Patterson”). RAID levels exist in a variety of standard geometries, many of which are defined by Patterson. For example, the simplest array, a RAID level 1 system, comprises one or more disks for storing data and an equal number of additional “mirror” disks for storing copies of the information written to the data disks. Other RAID management techniques, such as those used in RAID level 2, 3, 4, 5, 6 and 10 systems, segment or stripe the data into portions for storage across several data disks, with one or more additional disks utilized to store error check or parity information.
Regardless of storage management techniques, a mass storage system may include one or more storage modules with each individual storage module comprising a plurality of disk drives coupled to one or more storage controllers. In one typical configuration, a storage module may be coupled through its storage controller(s) directly to a host system as a stand-alone storage module. Typical storage controllers include significant cache memory capacity to improve performance of the I/O operation. Write requests may be completed when the supplied data is written to the higher speed cache memory. At some later point, the data in cache memory may be flushed or posted to the persistent storage of the storage modules. Also, read requests may often be satisfied by accessing data already resident in the higher speed cache memory of the storage controller.
Such direct coupling of a storage system to a host system may utilize any of numerous communication media and protocols. Parallel SCSI buses are common for such direct coupling of a storage system to a host. Fibre Channel and other high speed serial communication media are also common in high performance environments where the enterprise may require greater physical distance for coupling between the storage systems and the host systems.
Even in a standalone configuration, it is common to enhance reliability and performance by providing a redundant pair of storage controllers. The redundant pair of controllers enhances reliability in that an inactive storage controller may assume control when the active controller is sensed to have failed in some manner. The redundant pair of storage controller may also enhance performance of the standalone storage system in that both storage controller may be active—each acting as backup for the other while both simultaneously processing different I/O requests or different portions of an I/O request.
In such a configuration with redundant storage controllers, the storage controllers typically exchange information to maintain coherency of data between the cache memories resident in each controller. Some prior storage systems use the communication path between the controllers and the storage modules for the additional cache coherency information exchanges. However, such shared use of this communication path for interaction between the controller and the storage modules and for cache coherency exchanges between the storage controllers can negatively impact storage system performance. Some prior techniques have therefore provided a dedicated bus or channel coupled between the redundant pair of storage controllers and intended primarily for such cache coherency exchanges. Such a bus or dedicated communication channel is typically adapted well for simple, fast, point-to-point exchanges between the paired redundant storage controllers.
In another standard configuration, the storage module may be part of a larger storage network or cluster. In a storage network/cluster architecture, a plurality of storage modules and corresponding storage controller are typically coupled through a switched network communication medium (i.e., a fabric) to one or more host systems. This form of a multiple storage module system is often referred to as a Storage Area Network (“SAN”) architecture and the switching fabric is, therefore, often referred to as an SAN switching fabric. In such a clustered configuration it is common that all of the storage controllers exchange coherency information as well as other information for load balancing of I/O request processing and other control information. Such control information may be exchanged over the same network fabric that couples the storage controllers to the host systems (often referred to as the “front end” connection or fabric) or over another fabric that couples the storage controllers to the storage modules (often referred to as the “back-end” connection or fabric). Though such a fabric connection allows scalability of the storage controllers, use of the existing front end or back-end fabric may negatively impact overall storage system performance.
The differences between a stand-alone storage system and a storage network architecture are marked. In a stand-alone storage module system, a host computer system will directly send Input/Output (“I/O”) requests to the storage controller(s) of the storage module. The storage controller receiving the request, in general, completely processes the received I/O requests to access data stored within the disk drives of the storage module. The storage controller then identifies and accesses physical storage spaces by identifying and accessing particular logical unit numbers (“LUNs” often also referred as “volumes” or “logical volumes”) within one or more of the disk drives of the storage module. Via the storage controller, the host computer system can then read data from the storage spaces or write data to the physical storage spaces.
By way of contrast, in a multiple storage module configuration (i.e., networked storage or storage cluster), the various LUNs or even a single LUN can be spread across one or more storage modules of the storage system. In such a multiple module storage system the switching fabric may be used to effectuate communication between the storage controllers of one or more storage modules (e.g., via the back-end fabric) as well as between the storage controllers and the host systems (e.g., via the front end fabric). A host computer may communicate an I/O request to the storage system and, unbeknownst to the host system, the I/O request may be directed through the switching fabric to any storage controller of any of the storage modules. The storage controllers of multiple storage modules may require communications for exchange of cache coherency information and to coordinate and share information regarding LUNs that are distributed over multiple storage modules. Information returned by the storage controllers is routed back through the switched fabric to the requesting host system.
For any of several reasons, an enterprise may wish to change from a direct coupled storage module to a storage network/cluster architecture for coupling storage modules to host systems. For example, a network/cluster architecture may allow for increased available communication bandwidth where multiple host communication links may be available between the networked complex of storage modules and one or more host systems. Another potential benefit of a network/cluster storage architecture derives from the increased storage performance realized by the cooperative processing of multiple storage controllers that are interconnected to share the workload of requested I/O operations. Another possible reason for an enterprise to convert to a storage network/cluster architecture is to increase storage capacity beyond the capacity of a single, stand-alone storage module. The above-mentioned benefits and reasons may hereinafter be collectively referred to as storage performance features.
Any particular storage module has a finite storage capacity because, for example, a storage module has a finite physical area in which the disk drives may be placed. In addition, performance of the storage module may be limited to a number of possible controllers that may be configured within a stand-alone storage module for processing of host system I/O requests. Alternatively, a storage module may have a limit on the number of direct host communication links and hence a limit on the available bandwidth for communicating between the storage subsystem and host systems. Accordingly, when an organization requires improved performance features from its storage system, the organization may implement a new storage system designed with multiple storage modules in a storage network architecture to provide additional storage capacity and/or performance to overcome the limitations of a single stand-alone storage module.
Since a stand-alone storage module has a controller configured for direct access by a host computer system but typically not for cooperation and coordination with other controllers of other storage modules, implementation of a new multiple storage module networked storage system may include replacement of the storage controller(s) of the stand-alone storage module(s). Different storage controllers may be required to provide the required interconnection between storage controllers of the multiple storage modules to permit desired cooperation and coordination between the multiple storage modules. Such a reconfiguration of the stand-alone storage module is necessary because the storage module may coordinate with other storage modules through an SAN switching fabric not previously required in a stand-alone storage module.
Upgrades to an existing stand-alone storage system to enable networked communications among multiple storage modules remain an expensive process. In addition to possible replacement of storage controllers, retrofitting a present stand-alone storage module to operate as one of a plurality of storage modules in a networked storage system typically requires other components to implement communication between the storage controllers. Costly, complex N-way fabric switches add significant cost for the initial conversion from a stand-alone configuration to a storage network configuration.
Although storage performance feature requirements often grow in an enterprise, the cost for conversion to a networked storage architecture may be prohibitive to smaller enterprises. It is therefore evident that a need exists to provide improved methods and structure for improving storage performance feature scalability to permit cost effective growth of storage as an organization grows.